from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.315	0.108	0.0	0.018	1	100	0.828	2.139	0.008	8.561	0.059	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.315	0.108	0.0	0.018	1	100	0.828	0.021	0.000	877.241	21.485	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.009	0.0	0.304	1	100	1.000	2.139	0.008	0.142	0.004	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.009	0.0	0.304	1	100	1.000	0.021	0.000	14.550	0.554	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.214	0.057	0.0	0.018	-1	100	0.828	2.790	0.035	6.528	0.085	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.214	0.057	0.0	0.018	-1	100	0.828	0.025	0.003	723.969	80.538	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.003	0.0	0.304	-1	100	1.000	2.790	0.035	0.109	0.002	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.003	0.0	0.304	-1	100	1.000	0.025	0.003	12.066	1.347	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.229	0.008	0.0	0.018	-1	1	0.676	1.819	0.026	10.021	0.142	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.229	0.008	0.0	0.018	-1	1	0.676	0.024	0.003	757.432	94.330	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.306	0.007	0.0	0.306	-1	1	0.000	1.819	0.026	0.168	0.004	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.306	0.007	0.0	0.306	-1	1	0.000	0.024	0.003	12.727	1.609	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.288	0.074	0.0	0.018	1	1	0.676	1.157	0.006	15.800	0.099	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.288	0.074	0.0	0.018	1	1	0.676	0.020	0.000	909.761	17.115	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.007	0.0	0.303	1	1	0.000	1.157	0.006	0.262	0.006	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.007	0.0	0.303	1	1	0.000	0.020	0.000	15.058	0.454	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.116	0.108	0.0	0.018	-1	5	0.729	2.795	0.033	6.481	0.087	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.116	0.108	0.0	0.018	-1	5	0.729	0.025	0.005	711.011	126.448	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.004	0.0	0.307	-1	5	0.000	2.795	0.033	0.110	0.002	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.004	0.0	0.307	-1	5	0.000	0.025	0.005	12.057	2.149	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.359	0.189	0.0	0.018	1	5	0.729	2.128	0.010	8.629	0.098	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.359	0.189	0.0	0.018	1	5	0.729	0.021	0.000	882.359	21.799	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.007	0.0	0.304	1	5	0.000	2.128	0.010	0.143	0.003	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.007	0.0	0.304	1	5	0.000	0.021	0.000	14.628	0.468	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.616	0.011	0.0	0.004	1	100	0.895	2.080	0.002	1.739	0.005	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.616	0.011	0.0	0.004	1	100	0.895	0.002	0.001	1577.176	373.765	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.005	0.0	0.243	1	100	1.000	2.080	0.002	0.117	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.005	0.0	0.243	1	100	1.000	0.002	0.001	106.149	25.266	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.611	0.008	0.0	0.004	-1	100	0.895	2.695	0.021	1.340	0.011	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.611	0.008	0.0	0.004	-1	100	0.895	0.006	0.004	582.950	382.379	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.003	0.0	0.243	-1	100	1.000	2.695	0.021	0.090	0.001	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.003	0.0	0.243	-1	100	1.000	0.006	0.004	39.272	25.766	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.556	0.008	0.0	0.004	-1	1	0.854	1.675	0.027	2.123	0.035	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.556	0.008	0.0	0.004	-1	1	0.854	0.008	0.002	461.678	125.140	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.004	0.0	0.244	-1	1	1.000	1.675	0.027	0.146	0.003	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.004	0.0	0.244	-1	1	1.000	0.008	0.002	31.689	8.603	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.564	0.010	0.0	0.004	1	1	0.854	1.047	0.002	3.404	0.012	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.564	0.010	0.0	0.004	1	1	0.854	0.002	0.000	1969.260	180.162	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.004	0.0	0.243	1	1	1.000	1.047	0.002	0.232	0.004	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.004	0.0	0.243	1	1	1.000	0.002	0.000	134.185	12.447	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.561	0.008	0.0	0.004	-1	5	0.886	2.676	0.024	1.330	0.012	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.561	0.008	0.0	0.004	-1	5	0.886	0.009	0.003	399.184	127.469	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	5	1.000	2.676	0.024	0.090	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	5	1.000	0.009	0.003	27.150	8.680	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.559	0.010	0.0	0.004	1	5	0.886	2.083	0.022	1.709	0.019	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.559	0.010	0.0	0.004	1	5	0.886	0.002	0.000	1711.442	141.089	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	5	1.000	2.083	0.022	0.116	0.003	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	5	1.000	0.002	0.000	116.040	9.846	See	See

KNeighborsClassifier_kd_tree

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.186	0.000	0.0	0.119	1	100	0.967	5.176	0.024	23.025	0.107	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.186	0.000	0.0	0.119	1	100	0.967	0.005	0.001	25781.010	6527.269	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.613	0.095	0.0	2.613	1	100	1.000	5.176	0.024	0.505	0.019	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.613	0.095	0.0	2.613	1	100	1.000	0.005	0.001	565.223	144.578	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	112.595	0.000	0.0	0.113	-1	5	0.972	0.924	0.009	121.835	1.214	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	112.595	0.000	0.0	0.113	-1	5	0.972	0.003	0.001	33082.208	5638.378	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.604	0.059	0.0	2.604	-1	5	1.000	0.924	0.009	2.818	0.069	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.604	0.059	0.0	2.604	-1	5	1.000	0.003	0.001	765.210	131.551	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	117.244	0.000	0.0	0.117	1	5	0.972	1.528	0.006	76.734	0.315	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	117.244	0.000	0.0	0.117	1	5	0.972	0.002	0.001	62114.590	17328.828	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.609	0.111	0.0	2.609	1	5	1.000	1.528	0.006	1.708	0.073	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.609	0.111	0.0	2.609	1	5	1.000	0.002	0.001	1382.423	390.160	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.909	0.000	0.0	0.127	-1	1	0.949	0.464	0.006	273.427	3.313	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.909	0.000	0.0	0.127	-1	1	0.949	0.003	0.000	45274.396	7772.199	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.599	0.060	0.0	2.599	-1	1	1.000	0.464	0.006	5.600	0.147	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.599	0.060	0.0	2.599	-1	1	1.000	0.003	0.000	927.306	160.640	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.075	0.000	0.0	0.132	-1	100	0.967	2.911	0.021	45.375	0.320	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.075	0.000	0.0	0.132	-1	100	0.967	0.006	0.001	20504.183	4138.975	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.611	0.077	0.0	2.611	-1	100	1.000	2.911	0.021	0.897	0.027	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.611	0.077	0.0	2.611	-1	100	1.000	0.006	0.001	405.402	82.696	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.305	0.000	0.0	0.131	1	1	0.949	0.813	0.007	161.440	1.453	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.305	0.000	0.0	0.131	1	1	0.949	0.001	0.000	97867.133	31433.111	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.618	0.058	0.0	2.618	1	1	1.000	0.813	0.007	3.219	0.077	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.618	0.058	0.0	2.618	1	1	1.000	0.001	0.000	1951.231	628.180	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.004	0.0	0.000	1	100	0.896	0.033	0.000	1.819	0.115	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.004	0.0	0.000	1	100	0.896	0.001	0.000	98.797	23.401	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.033	0.000	0.159	0.008	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.610	2.012	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	-1	5	0.910	0.022	0.000	1.706	0.046	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	-1	5	0.910	0.002	0.000	16.169	3.334	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.000	0.237	0.009	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.244	0.467	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	1	5	0.910	0.020	0.000	1.901	0.030	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	1	5	0.910	0.001	0.000	63.707	14.519	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.020	0.000	0.263	0.008	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.818	2.022	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.036	0.001	0.0	0.000	-1	1	0.893	0.021	0.000	1.759	0.046	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.036	0.001	0.0	0.000	-1	1	0.893	0.002	0.000	15.673	3.170	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.021	0.000	0.252	0.009	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.249	0.458	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.001	0.0	0.000	-1	100	0.896	0.035	0.004	1.688	0.201	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.001	0.0	0.000	-1	100	0.896	0.002	0.001	27.495	6.895	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.035	0.004	0.148	0.018	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.001	2.413	0.609	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	1	1	0.893	0.018	0.000	1.985	0.040	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.001	0.0	0.000	1	1	0.893	0.001	0.000	61.862	15.182	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.018	0.000	0.281	0.008	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.769	2.161	See	See

HistGradientBoostingClassifier_best

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.460	0.017	0.002	0.000	0.823	0.104	0.001	4.414	0.166	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.460	0.017	0.002	0.000	0.823	0.016	0.001	28.604	1.585	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.376	0.012	0.000	0.376	0.000	0.104	0.001	3.611	0.120	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.376	0.012	0.000	0.376	0.000	0.016	0.001	23.400	1.229	See	See